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ABSTRACT 

The report describes the development of an 
alternative performance assessment procedure for certifying adults at 
the secondary school level in the New York External High School 
Diploma Program. Onlike standardized group tests, the alternative 
method is characterized by such features as flexibility in time and 
location of testing, variety of communication methods, and continuous 
feedback to students. The program's first developmental objective for 
1974-75 was a field test demonstration. Preparatory steps for the 
field test included defining performance criteria (translating 
generalized competencies into behavioral terms^ with accompanying 
performance indicators) , designing assessment components (developing 
diagnostic procedures and procedures for assessing generalized and 
individualized competencies) , and designing the operational form of 
the assessment system. Later steps included pre-testing the 
assessment system (selection and recruitment of subjects and 
observers, and method of testing) , and defining the role of the 
learning consultants (program development, counseling adult learners, 
and administering tests) . The second objective, the awarding of 
diplomas, included gathering information on the program's first 20 
graduates for later use in program revision* Objective three, the 
production of a final external diploma assessment package, will 
require additional time to accomplish. (JR) 



DEVELOPMENT OF A PERFORMANCE ASSESSMENT SYSTEM 
FOR THE CE.TKAL NEW .ORK EKTEKm HIGH SCHOOL DIPLOMA PKOGKAM 
M EDUCATIONAL ALTERNATIVE FOR ADULTS 



A PROGRESS REPORT 



us OEPARTMENT OF HEALTH. 
EDUCATION A WELFA8E 
NATIONAL INSTITUTE OF 
EDUCATION 

THIS OOCUWENT HAS BEEN REPRO. 
DUCEO EXACTLY AS RECEIVED FROM 
THE PERSON OR ORGANIZATION ORIGIN- 
ATING IT POINTS OF VIEW OR OPINIONS 
STATED DO NOT NECESSARILY REPRE- 
SENT OFFICIAL NATIONAL INSTITUTE OF 
EDUCATION PO'SITION OR POLICY 



by 

Ruch S. Nickse, Ph.D., Project Director 
and Coordinator of Assessment 
Regional Learning Service 
of Central New York 



Division o£ Syracuse University Reseatcn 1^ tepro- 
Syrac.se, Nev^iork 13203. No P^'^^/^/^f^^/^/.^i.^i.^ o£ the publisher, 
duced In any form without the prior written r 



PfRMISSlON to Mf^PMODUfF tHIS COPY 
RtGMTEO MATEniAl MAS 0F€N GPANTEO BY 



TO EniC AND 0''GA^4IZATI0NS OPfRATiNG 
UNDER AGRECMENTS Wtlh THE NATIONAL IN 
ST.TUTE Of EDUCATION FUfttHFM REPRO 
0<»CTlON OUTSIDE ThE EPiC SYSTEM RE 
OUIPES PERMISSION Of THE COPYRIGHT 
OWNER 



V 



ii 



ACKNOWLEDGEMENTS 



ERIC 



This Progress Report gratefully acknowledges 
the creative efforts and energies of the 
RLS Assessment Staff 
Linda Lehman - Jean Kordalewski 
Margaret Charters, Ph.D., Judy Alamprese, Marion Stone, R.N. , 

and 

RLS Personnel 
Learning Consultants 
Francis Macy - Donn Vickers 
and 

Susan Hanley, Rosemary Halko, Vikki Preston, Cynthia Sanders 

and 

Jacqueline Dedell 



iii 



TABLE OF CONTENTS 

Page 

iv 

SUMMARY 

1 

INTRODUCTION 

1 

HIGHLIGHTS OF PROGRAM 

4 

PROGRAM OBJECTIVES: 1974-1975 • 

A. Objective One Field Test Demonstration ^ 

1. Definition of Performance Criteria ^ 

2. Design of Assessment Components 

.... 8 

a. Diagnostic Procedures 

b. Assessment of Generalized Competencies | 

c. Assessment of Individualized Competencies 

(1) Occupational/Vocational ^ 

(2) Advanced Academic 

(3) Advanced Specialized 

3. Design of the Operation of the Assessment System 1^ 

4. Pre-Test of Assessment System 

a. Selection and Recruitment of Subjects 

b. Selection of Observers 

c. Method of Pre-Testing 

20 

5. Role of the Learning Consultants 

20 

a. Program Development 

b. Counseling Adult Learners ^| 

c. Administration of Diagnostic Testing 

22 

B. Objective Two — Awarding of Diplomas 

C. Objective Three - Production of a Final External Diploma 

Assessment Package 



iv 



SUMMARY 

On March 11, 1975, Commissioner Ewald B. Nyquist presented high school 
diplomas to the first eighteen graduates of the Central New York External 
High School Diploma Program. The ceremony, held at the State University at 
Cortland, was attended by some two hundred persons who have been involved 
in the program since its inception, families and friends of the graduates. 
Commissioner Nyquist, in describing the program, called it '^humane and 
relevant" and hoped for its adoption throughout the state. 

Ihe graduation exercise was a milestone in the developmental effort 
which was begun by Stephen K. Bailey in 1972. The event was a happy and 
significant pause in the on-going demonstration of the educational innova- 
tion for adults, and the ceremony itself was a concrete summary of the pro- 
gress to date on the project. 

The following report describes the project's progress from August 197A 
through April 1975. It contains a description of the unique characteristics 
of the program and the specific details of the design and implementation of 
the demonstration and assessment system. The report presents the steps from 
conceptual design to an operational procedure which, although currently in 
practice, represents only a phase in the logical completion of the demon- 
stration. If we are to answer essential questions about feasibility and 
replication, and bring the demonstration to completion, it is required that 
we continue to operate the program until 1977. 



DEVELOPMENT OF A PERFORMANCE ASSESSMENT SYSTEM 
FOR THE CENTRAL NEW YORK EXTERNAL HIGH SCHOOL DIPLOMA PROGRAM 

I. INTRODUCTION 

The purpose of this report is to trace the progress made by the 
assessment staff of the Regional Learniitg Service of Central New York (RLS) 
in the development of a performance assessment procedure for certification 
of adults at the secondary school level through an external high school 
diploma program. The aim has been to create a new diploma program 
specifically tailored to the needs and experiences of adults in twentieth 
century America. 

The demonstration, funded by the Ford Foundation and by the New York 
State Education Department., is being conducted by RLS, a division of the 
Syracuse University Research Corporation. Dr. Ruth S. Nickse, Project 
Director, is assisted by Ms. Linda Lehman and three part-time persons 
assigned to the assessment staff. 

II. HIGHLIGHTS OF PR OGFJVM 

The Central New York External High School Diploma Program, guided in 
its design by a set of assumptions about adult students, is essentially an 
assessment program quite different from familiar standardized machine-scored 
testing programs. 

The development of this new performance assessment system was predicated 
on the RLS understanding of the need felt by many adult learners for a testing 
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situation responsive to their particular abilities and concerns. It was 
anticipated that many potential diploma candidates would have had disappoint- 
ments ana failures in standardized group testing situations and could, there- 
fore, be expected to be highly text anxious; perhaps cynical and rebellious 
about testing which was irrelevant to their concerns as adults; very busy 
managing full-time jobs and families; highly motivated to work for a diploma 
after years out of school; responsible; and in need of an opportunity to 
choose assessment modes which would best enable them to present their skills 
and competencies. These assumptions about potential candidates led to th" 
development, of an assessment program which is unique. Some key features are 
summarized below. 

Flexibility in Time and Location . The system's self-paced characteris- 
tics provide the opportunity for busy adults with heavy responsibilities to 
establish some of the conditions of testing. Three out of five assessment 
tasks can be completed at home, or, if preferred by the student, performed 
in the RLS office without time limits. This flexibility in timing, as well 
as some choice of environment (home or at the RLS office) offers testing con- 
ditions designed to reduce test anxiety and improve performance. 

Open Information on Requirements . The competencies which must be 
demonstrated for successful completion of the program are made explicit and 
are discussed freely during the counseling and the final assessment procedure. 
This openness of the test content represents a departure from familiar test 
procedures and is consonant with the structure and philosophy of the program. 
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Breaath of_ Skills and Knowledge . In addition to comprehension and 
recall skills, the program rewards occupational skills and process skills 
with emphasis on the lattei . Many of the items in the tasks call for 
application of cognitive skills of synthesis and analysis, rather than 
requiring the more familiar, and lower order thinking skills of compre- 
hension and recall. 

Variety of Communication Modes . The take-home projects require pre- 
sentations of student's work in written form, and provide opportunity for 
those with strong writing skills to present them in this fashion. In 
addition, two oral performance interviews encourage persons who may feel 
more comfortable in expressing themselves and what they know through speech. 

Flexibility of Assessmen t. Process skills, such as accurate math 
functions, which have only one ''right" answer, of course, must be answered 
with 100 percent accuracy. Many other items, however, have a broad range 
of acceptable answers, e.g., items which ask for a person to present infor- 
mation about the location, use and comparison of community resources. Items 
Chat require documentation or other proofs can be supported in a variety of 
acceptable forms. 

Continuous Feedback to Students . Progress through the assessment 
prograr. is transmitted at each step to students through written reports 
mailed to their homes. Assignment tasks are not "failed*' or "passed." 
There is no "passing score." Competencies are simply demonstrated or 
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not demonstrated and each competency is so noted in :he task performance 
report to the student. If a competency has not been demonstrated (\t is 
either incomplete or incorrect) an explanation of the error is included. 

Unlimited Opportunities _to Demonstrate Competencies . Students may 
attempt as often as they wish to demonstrate those competencies they were 
unable to exercise adequately on the five basic assessment tasks. Since 
clearly defined competencies are being assessed, there is no need for a 
learner to repeat an entire task when a particular competency has not been 
demonstrated. 

III. PROGRAM OBJECTIVES - SEPTEMBER 1974 TO APRIL 1975 

The objectives of the program during Phase III, covered by the con- 
tract September 1, 1974 - December 1975, was three fold: (1) to field test 
(demonstrate) all components of the model for an external high school diploma, 
(2) to award diplomas to those who demonstrate the competencies which con- 
stitute the content of the external diploma, and (3) to produce the final 
text of an assessment package that would be suitable for use by local school 
districts and the State. Substantial progress has been made on objectives 
one and two. The field test with actual diploma candidates began formally 
on December 1, 1974. By April 1975, 20 adults have received high school 
diplomas. Objective thiee activities are underway and will be discussed 
later. The progress to date on the above objectives follows. 
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A. Objective One - The Field Test Demonstration 

All components of the program had to be pre-tested on paid sub- 
jects in order to prepare a legitimate demonstration of an external high 
school diploma program. This initial pre-testing, vhich would closely 
approxin.ate the procedures to be used during the subsequent demonstration, 
was Qesigned and implemented during the period from August 197A through 
November 30, 1974. The only valid demonstration, however, is with actual 
diploma candidates. Field testing with candidates began December 1, 197A. 

Before field testing could be conducted, a number of major prepar- 
atory steps were completed: definition of performance criteria, development 
of the assessment components (diagnostic and final testing instruments for 
assessing Generalized and Individualized Competencies) and operational design 
of the assessment system by which the demonstration would be conducted. 

1. Definition of Performance Criteria 

When an assessment procedure is developed, a value is placed on 
certain knowledges and skills. Therefore, the precursor of any assessment 
system is the identification of the content area to be tested. For this pro- 
ject, identification of content meant the translation of the Generalized 
Competencies selected by the Design Group into behavioral terms, with accom- 
panying performance indicators, which could be the basis for the development 
of new assessment procedures. 

Accordingly, in October 1973, a task force of persons was 
assembled to define the minimum competencies. The fourteen persons were 
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selected according to the following criteria: 

1. Knowledge of the secondary school curricula. 

2. Prior teaching or work experience with adult learners. 

3. Broad life experience. 

4. Ability and prior experience in writing behavioral objectives. 
Two assumptions, based on familiarity with the needs of adult 

learners, provided a working framework for the task force: 

1 That although the acquisition of learning may be discrete, 
its application is wholistic. Therefore, competencies in 
the tool subjects (communication and computation) as we.i 
as in Life Skills must be survival-oriented and demonstrable 
in context. 

2. That coping adults in a changing society must be able to 
use process skills such as data gathering, manipulation 
and application in problem solving with information that 
is personally and socially relevant. Therefore, com- 
petencies and the performances by which they are tested 
must be related to the common developmental tasks ot 
adulthood . 

The fourteen developers, working in small groups, across and 
within disciplines, identified and selected by consensus sixty-four competencies 
which they felt responded to this question, "What skills do adults need to cope 
with daily living?" Whea a preliminary list of competencies had been identi- 
fied, persons from small business, industry, unions, schools and social agencies 
were asked to react to each competency. Their written comments and their rat- 
ings of the importance and rolevance of the competencies were provided as 
feedback on which the development group based further selection of the com- 
petencies and refinement of the performance indicators. 



^See Appendix A, Competency Development Task Force. 



11 



FRir 



7 



On Marcr. 14, 1975, the third draft of the competencies list was 
approved by the Regional Committee for the External Diploma Program, aid 
subtr-itted by Its chairman to the State Education Department for its approval 
of a pre-test to be ccnducted through December 1974. The Department, on 
August 21, 1974, instructed the Regional Learning Service to proceed with 
field testing of the competencies assessment system. 

2 . Design of A ssessment Component s 

A new assessment program offering adults an alternative system 
ot demonstrating real-life competencies in practical settings was the goal of 
the assessment design effort. A battery of standardized instruments, noi.-^d 
on youngsters in high school classes, obviously would not produce the sought 
for ^.ItHrnatives nor the opportunity to apply the competencies to the practical 

problems faced by adults. 

Therefore, the assessment team developed a new assessment system 

with the following objectives: 

1. To give some control over the testing environment to the 
candidates. This was accomplished by designing the process 
of diagnosis and assessment to be initiated on demand by the 
candidates and self-paced. 

2. To make the assessment process a learning experience. 
Thus, candidates are f\iliy informed of requirements and 
their performances at each step. The testing process 
emphasizes syi.thesizing and application, rather than 
simply knowledge and con.prehension. 



* 

See Appendix B. 
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3. To relate assessment in form and content to the con- 
cerns o. adults. The content, therefore, consists of 
projects based on the practical developmental tasks of 
adulthood (changing residence, applying for a job, con- 
sumer awareness, caring for family - "al health). 
These projects are performed as take-iiume simulations and 
as oral interviews. This process permits flexible 
scheduling for completing the work. 
4. To make the testing process humane. Thus, the procedure 
is individualized, allowing candidates to assume much 
responsibility while providing them with continuous feed- 
back and success experiences • 

a • Diagg c • ^ Ic Procedure s 

The RLS assessment staff decided to include, at the outset 
of the assessment process, a diagnostic step. This provides candidates with 
(1) concrete information about the competency required, (2) supportive Lecd- 
back on the many competencies the> already possess, (3) specific information 
on the competencies they still need to develop, and (4) experience in modes of 
assessment they did not encounter in school. Basically, RLS wanted to give 
candidates a feeling of confidence while helping them to overcome any feelings 
of inadequacy and failure. 

Not all competencies could be tested by available commer- 
cial instruments so the staff designed several new diagnostic inctruiuGnts. 



^Haveghurst, Robert, Development al Tasks and Kducatio n (Third Edition), 
David McKay, New York, 1972. 



Instrume nts Designed RLS 

, Self- Assessmen t Ch ecklist - RLS. A self-administered checklist 
h familiarized candidates with the Generalized Competencies 
which iihey will be asked to demonstrate in the final assessment. 
It asks them to rate their own abilities, primarily in the Life 
Skills area. 

• -:^.ath Diagnostic - RLS. A math diagnostic based on the required 
math competencies which must be demonstrated in the final assess- 
ment procedure. 

. Wri ting Diagnos tic - RLS. A writing diagnostic, based on writing 
competencies which must be demonstrated in the linal assessment 
procedure. 

Some commercial Instruments were bufficient for diagnostic 
purposes in the program. 

Coi?jrierc ial Inst rumen t^ 

. Rea ding/Eve ryday Activities in Life (R/EAL ) - Lichtman. A standardized 
reading test developed by the Job Corps for use with adults. It is 
self-administered and based on comprehension of materials common to 
adult reading tasks (maps, sets of directions, job applications, matrix 
reading, leases). 

. Self- Direct ed Se arch - Holland. A self -administered inventory for 
use in determining occupational intf»rests and abilities. 
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b . Assessment of Generalized Competencies 

Tne assessment team developed five assessment instruments 
or tasks for the assessment of Generalized Competencies. Each task provides 
a framework within which the candidates demonstrate a variety of generalized 
competencies in fulfillment of requirements for the diploma program. 

A brief description of the five tasks follows: 
TASXl: CHANGE OF RESIDENCE • This task of selecting 
a new apartment through the want-ad section of the newspaper provides an 
opportunity for the demonstration of many widely-used competencies. Computa- 
tion, measurement, reading, writing, map skills are tested. 

TASK II: PERSONA L AND FAMILY HEALTH - An oral interview 
by a public health n.rse, using a structured checklist, elicits competencies 

in personal and family health and safety. 

TASK III : OCCUPATIONAL PREPARLDNE SS - Job seeking skills 
are demonstrated through such activities as comparing and classifying jobs 
advertised in newspapers, writing a resume and letters of application for 

jobs, and identifying payroll deductions. 

TASK IV: PERFORMANCE INTERVIEW - A trained assessor poses 

c-n 1 1 c anA in listening and viewing, 
problems in measurement, conversion, map skills and m ii.sLen b 

A writing exercise on the topic "Why I Need a High School Diploma" becomes t.e 

basis for a semi-structured conversation concerning the students' values, needs 



and i,oal3. 



TASK V: CONSUMER A WARENESS - This task is focused on 
applying problem solving skills to consumer purchases. The candidate is required 
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^ ^u--''ic an thev are related to 
to demonstrate rea=.nin8, readxng and writing sk. .is as tney 

good consumer practices. 

I,u,s combination o£ three take-hone projects and two oral 

as.lsn.ents proviae. the opportunity £or aemon.tratlon of the sixty-£ocr com- 
petencies deemed as minim™ re,uir«.nts In computation, communication, and 
other Life SUUs. Successful performance of these competencies is re,uired 
of all students. The tasKs are not sequential and can be performed in an. 



order, 



c. ^-^-j;:^ nf Individualized Competencies 

In addition to demonstrating a com:non set of mini>nun, 

mMc^ Al<;n orove possession of 
Generalized Competencies, each candidate must P 

advanced competencies in an area of his or her choice. 

There are at present three areas in which students may 
request to complete assessments in Individualized Competencies. This allows 
Wide Choice for students, as they must meet requirements in only one of the 
three possible areas and within each area there are choices to be made. The 
areas are: 

Occupational/Vocational. 
. Advanced Academic (for college preparation). 
. Advanced Specialized Competencies. 

Each area identified within the individualized Competencies 
presents a need for a different assessment technique. 

(1) nnnuoational/Vocatiojial (entry level job skills). 
There are two alternatives available to candidates should they choose assess- 
ment in tl^is area. 
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. Employers Testimony . This assessment procedure 
is designed for candidates who have worked for a year or more at the same job. 
Documentation in the form of a questionnaire filled out by an immediate super- 
visor attests to the on-the-job duties and responsibilities of the candidates. 
A worker characteristics checklist, also filled out by the supervisor, rates 
the candidate on non-skills characteristics which are important to that par- 
ticular job. Satisfactory documentation, through employers testimony, con- 
stitutes meeting occupational/vocational requirements for the diploma program. 

Performance . This assessment procedure may be 
requested by candidates who have occupational/vocational skills, but have 
not been employed for a years' time, who are self-employed, or who are 
employed in a family business. Judgment of specialists in the particular 
occupational/socational area is sought in evaluating entry-level job skills 
of the candidates. The RLS assessment staff locates a Master Craf tsperson , 
one who is employed as a professional in that field, and arranges for a 
performance assessment to be conducted insofar as possible in an on-the-job 
setting. A.--eement on criteria for judging entry-level job skills in each 
field must be determined by consensus of experts, and great variations exist 
even among experts in the same field as to what consitutes entry-level criteria 
for jobs. Therefore, the RLS assessment staff carefully selects an appropriate 
craftsperson to perform the assessment and accepts, as well as collects, the 
criteria by which the candidate is evaluated. In this manner, criteria for 
different jobs, expressed by specialists, are collected and analyzed by the 
FlS assessment staff for future occupational/vocational assessments. 
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. New X2Ei SHH Licensin g. The assessment staff 
IS investuatm, the potential of accepting cuttent .e„ Yot. State licenses 

occupational fields as evidence of occupational competence. An acceptable 
current license would constitute satisfactory documentation, and would fulfill 
the occupational/vocational requirement. 

(2) Advanced Academic. This assessment procedure was 
aeveloped from the suggestions of admissions administrators at several local 
two- and four-year colleges. Called the Advanced Academic TasR, it requires 
.andidates to demonstrate a readiness for college level wor. hy requiring them 
to present several Kinds of evidence. The evidence requested involves auto- 
hiographical essays which outline candidates^ past and present educational 
aims, as well as a feasihility analysis of the particular problems candidates 
might encounter in college attendance. Some evidence of ahility to do college 
..or. is required. Th.s evidence may he of two Un.s, presentation of some 
original wor. which indicates college level ahllities. or an acceptable score 
on standardised tests in math and reading competencies. The Iowa Test of 
Educational Development in math and reading has been selected by the assos.,- 
ment team as an appropriate instrument for indicating readiness for college 
level wor.. This test, although somewhat at odds with the assessment philosophy 

In rhP absence of other suitable assessment pro- 
of the program, has been used in the absence o 

other acceptable criteria for ma.ing this Judgment are under exploration by 

the asbcbsment L»'.aff. 
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(3) Advanced Spec ialized- This assessment procedure, 
designed primarily to enable persons to demonstrate a strong avocational skill, 
presents a similar assessment problem as that of the occupational/vocational 
area, and is handled by the use of judgment of specialists. 

3. Design of the Operation o_f the Assessment Sys^ 

The organization of the assessment procedures from program entry 
to program exit can be summarized in eight steps. The diagram which follows 
presents the eight steps as they would be taken uy any student. 

Step 1 : Students are assigned RLS learning consultants who ..es- 
cribe the alternative diploma programs available (e.g., high school equivalency, 
twenty-four community college credits, evening high school). 

Step 2; For students who select the external diploma program, 
diagnostic pre-testing is administered by the RLS learning consultant (see 
description of diagnostic instruments. Section 1). 

Step 2A : If diagnostic pre-testing exposes deficiencies in com- 
petencies, learning prescriptions are issued, and students enter necessary 
learning phases. These may be brief or extended, depending on the deficiency. 
Students are informed by RLS of the places and persons in the community where 

competencies can be learned. 

Step 3 : If diagnostic pre-testing reveals that competencies are 
at the required level, the student immediately begins the assessment of Generalized 
Competencies (Tasks I - V described above) . Since candidates could receive help 
from others on the tasks they complete at home, they arc required to demonstra.- 
certain key competencies to the assessment staff. The competencies to be spot- 
checked are known in advance by the candidates. 
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Step_A:, The individualized assessment area is picked by students, 
and the processes are initiated to fulfill these requirements. Step 4 occurs 

concurrently with Step 3. 

Step 5 : The post-task assessment is the occasion for the demon- 
stration of Generalized Competencies not successfully demonstrated in Tasks I - V. 
Post-task assessments may be repeated until competencies are demonstrated. 

Step 6 : The RLS assessment teaTn constantly monitors students' 
progress and issues progress reports to students, as well as maintaining records 
and assembling students' portfolios. Upon their recommendation, completed port- 
folios are submitted for review by the Regional Committee. 

Step 7= The Regional Committee, composed of 16 local persons from 
education, business and industry, meet monthly to review the portfolios which 
contain the records of student performance in assessments, to approve them, 
and to recommend awarding of diplomas for successful completion of requirements. 

Step 8 : On the recommendation of the Regional Committee, the 
Cortland, New York, City School Board awards Cortland diplomas to successful 
students on a bi-monthly basis. Special circumstances may result in a more 
frequeut action on the part of the Cortland School Board. 

A Procedures Manual for the assessment system was developed as 
a guideline tor use by the learning consultants and staff. U sets forth all 
steps in the assessment system: diagnostic testing, task taking, feedback and 
records maintenance. It also describes the roles of the student, the learning 
consultant, the RLS assessment staff (two full-time, one part-time) and the 
part-time assessors (two). 
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External High School Diplcana Progren 



STEP 2 



j Diagnoatic Testing^ 
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STEP 3 



Assessment of 
Individualized 
Competencies 



Assessment cf 
Generalized Competencies 



Tasks 1>3>5 
(Take-Home) 



Spot Check 
(at RLS) 



Tasks 2^k 
(Interview 
at RLS) 
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STEP 5 



Covcnmity 
College 



STEP 2A 



Learning Phase 
to Acquire Heeded 
Competencies 



STEP 6 



Post-Task Assessment 
of Unc^emonstrated Competences 
(RLS Office) 



Candidates Recommended hy 
RLS Assessment Staff 
to Regional Committee 



STEP 7 



Rcviev and Approval 
of A.'^sessments 
hy Regional Committee 



STEP 8 
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\V Cortland Enlarged City School District 
Cortland, Hew York 



17 



^' P re- Test of Assessment System 

Both the test instruments (tasks) and the process itself (of 
tdke-home projects, oral assessments) were pre-cested by paid subjects in 
the fall of 1974, during the months of September, October and November. A 
shore description of the pre-test follows. 

a . Selectio n a nd Recruitment of^ Subjects 

The paid subjects were selected to be a.s similar tu the 
anticipated target population as possible in age and ethnic background. AM 
had high school diplomas, but no further education. The selection of subjects 
with high school diplomas was desirable. If credentialed adults had these 
skills, t.en others seeking the same credentials could legitimately be expected 
to demonstrate them. 

b. S election of Observers 

Because information on both content and process of the 
assessrtient program was needed, trained observers with experience in observa- 
tion techniques were recruited from graduate students of Syracuse University. 
They were told by the assessment staff to act as "private secretaries' to the 
subjects, recording all activities and feelings expressed by subjects as tr.ey 
worked through each task. 

c. M ethod o± P re-Testing 

As each task was developed, subj ect/ol server pairs were 
scheduled to meet with a member of the assessment staff for a briefing session 
ich included getting acquainted and establishing rapport. The pairs then 
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. ^il the task was cotapleted. under conditions which would 
worked together until the tasK was h 

. . . 1^. in a self -directed, take-home project. 

hhn<;p expected of adults m a sexi. 
approximate those expt:i-i-'= 

. of rhp subiects, and accompanied 
..-^prf 11 actions and comments of the subject , 
Observers reco'f?«'d di action 

. for successful completion of the task. When the 
,,em wherever necessary for ^^^^^^^^^^ 

subject finished a task, a debriefing occurred and the subj 

.Sights as to clarit. of directions, format of task, misconceptions an 

... 3ubJect-ob.erver pairs until directions and Procedures wer. clea.l. 
^^^^^^ ... tasks designed to elicit the Cenerali.ed Competences 

was test, in this fashion, the adult subjects. 

Objects were able to complete each of the tasks providing continuity a con 

KeorvPrs were used throughout the pre 
^- «o Also the same observers weit 
Qic;tencv of reactions. Aiso, tuc 

1 ... ..... ...e.e„c.. ..... . n-e. 

feedback Co acco.pUsh revisions o£ task instruments. 

Xne sa„e type of subject/observer interaction vas used 
,e-test processes in tHe Xndividuaii.ed Competencies areas, but because o. 
,.e pressures, oni, one of eac. t,pe of Xndividuaii.d .sess.ent procedure 

^e^nns also but more experience 
„as attempted. These resulted in process revisions 

with this portion o£ the diploma is needed. 

OccuEtional/Vocational As-S?« °' ^""^ 

„eat cutter. This included location of an appropriate master butcher, an 
,„.ervie„ to dete^ine assessment conditions, and then an actual meatcuttin. 
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• , n r on-site by the master butcher, the subject 
perfornauce test, carried out on-site by 

• . the observer. Fro. this experience, criteria for entry-level 

apprentice, and the observer. 

1 ^ T1- is anticipated that the criteria 
► ►•!r,o fierp pvolved. It IS anti^-j-F"'- 
iob skills in meat cutting were evoi 

..US a. .0. U.e. .e e.^Us.. . ..P^o^a P"."" 

...... .he .Cel „hicH viU .e usea whenev. i. is necessa.y 

■ 4 n«n of a years' satisfactory 
assess occupational/vocational competences In Ueu of a y 

. The process itself seems to be a<lco,uate but vUl be 

experience on the job. Ttie proces 

adjusted in light of experience. 

Tbe Mv-ced Academic area, wbicb was defined by admls».ons 
,,.ctor» of several local colleges and junior colle.es. also pre-testcd 

•d subject The individual, following the outlines of the college 
with one paid subject. 

. ..bled a college readiness portfolio and also took stan- 
readiness task, assembled a coiieg 

c;i,p nresented this portfolio 
dardi.ed tests In math and reading competences. She presen 

KHr two-vear institution and at a four- 
to local admissions officers at one public two year 

• for discussion. Both administrative officials pro- 
year private institution for discussion. 

. -^.ff as to the adequacy of the materials 
vided feedback to the RLS assessment .taff as 

which will guide the continuing refinement of this process. 

. S^^ializ^d Com^^ie^ ^^ssess^,,. in the area of art 

J c;n^.riAlized Competencies area. The 

^ or.H for recoanition in the bpeciaix-ceu 
for judgment and tor recuguxux 

.^o,- -.rr-^nped for an on-site visit 
*- er^ff located a commercial artist, arrangea 
RLS assessment statr locaLcu 

u ^V,P nroducts were evaluated and some useful criteria 
to tiie studio, where the products were 

were delineated for co^ercial art job skills at entry-level. 
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AS a result of the pre-test o£ the five tasks designed 
to elicit Cenerali^ed Competencies and the pre-test o£ the three areas of 
.He lndlvid„ali.ed Competencies, the follovlns were revised and Judged adequate 
use .ith candidates. AU Instruments and processes, however, are considered 
the first generation stage and require data from actual use to refine them 



properly. 



1. Diagnostic pre-test instruments. 

2. Final assessment tasks (Generalized Competencies) 

3. Final assessment tasks (Individualized Competencies 

4. Spot checks. 

5. Post-task assessment procedures. 



5. Role of the RLS Le_arnin^ Con_su.ltant 

The role of the RLS Learning Consultants in Phase III of the 
external diploma project has three dimensions: (a) pro.ram development, 

counseling of adult learners, and (c) administration of diagnostic testing. 

a. ^^2M3^^ De velopmen t 
' uarnTng consultants have assisted in the development of the 
program, principally in providing feedhac. In the design of operations and pro- 
cedures of the assessment system. A ten-step procedures manual, which is used 
hy the learning consultants to counsel individuals in the program is a result 
of training sessions held hy the assessment staff and tV. ..arnlng consultants. 
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b. Comiselin^ Adult Learners 

Uarning consultants have counseled aduU learners who have 
expressed a ne3d to ohtaln a high school dlplo.a. Learning consultants i.part 
detailed Information on the four alternative paths to the high school credential 
in Ne. Yor. State (the CED, adult high school programs, the 24-hour co^unrty 
..Uege ro..te and the Central New YorK External Blplo.a progra.,) • Individuals 
Choosing to worK for a high school credential through the external diplo^ pro- 
gram h,ve been assigned learning consultants who have explained the program 
details to them, and provided support and encouragement. 

c . Adminlstr at Ion of Plaanosttc TeiilOS 

Learning consultants have attended several training sessions 
Which have equipped them to administer diagnostic testing for the externa, 
"diPlona orogram. This step in assessment provides the learning consultants 
.„d the adult learners with specific knowledge of their strengths and weak- 
nesses before they begin the final assessment process. The final assessment 

^ cc: n^iT-ino fhe final assessment 
process is administered by the assessment staff. During the 

phase, the learning consultant's role is largely supportive. 

The three dimensions described above have con,,tituted the role of 
rhe learning consultants in the external diploma program throug,, the development 
Phase of the project. The experience to date is not representative of the role 
of the learning consultant in any replication of program. No replication would 
need to utilize persons such as learning consultants for development of procedures, 
as they are now complete. 
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The principal function of the learning consultants no« in th« 
demonstration are counseling an. diagnostic testing. U is not clear at t,us 
early ,ta.e of the demonstration how much counseling «iU he necessary in any 
.eplication. This is one of .he questions to he studied a. the field test 
demonstration continues. Data is currently being gathered towards this end. 

B ObjecUve T»o - Awarding of the. DiElomas. 

I, Oecemher 1. 1974, the system was complete enough to begin assess- 
ing students, and on January 28. 1973. the Cortland School Board unani„us.y 

^ fir<zr ten ^Taduates who had 

voted to award its l.igh school diploma to the fxrst 

successfully completed the program by that date. 

on March 11, 1973, Commissioner Nyq.ist officially presented the 
Cortland City School District diplomas to the first eighteen graduates of 
program. The ceremony, held at SUN. Cortland, was attended by several 

.en. Che program, and by relatives and friends of the graduates. 

The awarding of a Cortland City School diploma occurred as a res.lt 
3everal presentations given by the RLS assessment s.aa to the Copland 
,.ard of Education and communication.s from the State Education Department, 
on October 3, 1.., a Department official requested the Cortland City School 
Soard to award its diploma to individuals throughout the five counties .n 

, V, niPte the ex-rnal diploma program. On October 22nd , 
Central New York who complete the ex rud± p 

- ,iv voted to do so when candidates are recommended 
Cortland Board members unanimously voted to 

by the Regional Committee. 
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The Regional Committee for the External Diploma Program has con- 
tribiaod am! continues to contribute to its successful demonstration. The 
motnbors . f the Regional Committee act as tha school board for the diploma 
process; Mem-^ers include educators, administrators, business and union 
representatives from throughout the Central New York area. They meet 
monthly ror three to four hours with the assessment staff to review and 
monitor its work. 

The Committee members, who volunteer their services, read and review 
the portfolios which contain all of the work completed by each of the car.!: 
dates for the diploma. They formally vote to recommend that the Cortland 
City School Board award its diplomas to the candidates under review. Coi^nittee 
members nake policy decisions concerning many aspects of the program as well. 
They have, as occasion demanded, served on sub-committees to assist in re- 
design of the procedures manual, to plan the graduation ceremonies, and to 
fill, with appropriate rr.embers, vacancies on the Committee. 

They also act as liaison between RLS and the State tiducation Depart- 
ment, and report to the Joint Management Team of regional educators. Because 
of their business, professional and personal affiliations, the members are 
spokespersons for the program on many community levels. Their conunitment, 
high level of enthusiasm, and participation is an essential component of the 
assessment system and has contributed significantly to the successful comple- 
tion ot Che program by the candidates. 

Demographic information is gathered on the persons engaged as candi- 
dates in the program and is incorporated into case studies of the individuals 
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system to the assessment team. 

T;nT-cf Fichteen Gra duates 
Information on I-irst LiftnLe cu ^ ^ 

19 . 61 years (average 34,5 years) 
Age Range J 

9 men, 9 women 
9 black, 9 white 

Race 

U ..id-April. there a.e U persor. In t.e pro„r». In va.ions 
,.3. . c«.nseun„ .ia.nes.U ce.Un.. lea.n.n.. o, Hn,a as.,ess„=n. On 

,• „ , are taking less than three months to complete the 
th^ average, individuals are tamng le 

assessment syste. which is self-paced and self-iniciaced . 

. n, an extensive post-graduation interview schedule has 
the assessment system, an extensiv h 

A rn all eraduatcs of the program by 
•oeen designed,, and is being administered to all gr 

... staff This information, which is enthusiastic.U> 
members of the assessment statt. 

Koo-ic, for program revision 
reported bv the recent graduates, will become the basis 

. r- ent A recent graduate o£ the program will he asued to become a 
and refinement. A recenu & 

.ember . the aeglonal Committee. In addition, a onc.ear .oll»-up interview 
,s Planned to determine how having earned the high school credential has 
a££ected the educational or job goals o£ the graduates. 
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. . Obloctive Throe - Production of a Lin,l 

Diploma As sessment Packajge 

succoHsta ..»pletlo„ of the program by 20 grad.ntes by »,ld- 
U a »ajor achiove».„t. U aces not provide, ,.o„=vet. an adequate 
.„et.e„c. and data base on which r.o produce the definitive te..» of a co„- 
puceo. a.3„.™e„t paCage, including cost analyses, personnel and training 
.cco^cndatlons, and manuals for assessors and clients. 

had planned to test the assessment procedor.-s in a school .cU.ns 
ars. needs .o.e experience with its own administration of the progra.. The 
present ..ntract with the department will enable the demonstration to continue 
en.v through ,,une 1,75. ,<LS will, therefore, request an ..tension of the .on- 
.,„et to continue the demonstration In several settings Tor two years to collect 
Ue data necessary for replication of the external diplo„ program throughout 
the State of New York. 



TV ^^^'iC:lus_ion 

;,;^,'7;port has described the substantial progress n,ade on objectives 

i„ no means complete. Completion of the program, including alt 
.o.,>onents necessary for replication, will be accomplished when the following 
objectives have been attained. 

^TQ pffort to Cortland, possibly Oswego, 
1. Decentralization of RLS eltort no 

and Auburn. 

Demonstration of program in other environments. 
Revision of assessment package based on experience. 
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K= ancuered orior to recoramendation for 
Questions which remain to be answerea prxui 

tr.ip 1 i>-.iC ion Lnc luclc : 

1 . 'Aliat are the costs of replication? 

2. '.vrhat are the possible sites for replication? 

3. ;>nat are the personnel and training technique, necessary to 

implement the program? 
This educational alternative has had, in it,, short Ure, a demonstrable 
i.„act on tne lives o£ B0.e 20 adults vho have .o.plctc,i it and another .0 still 

tn th. pro.ra™. It is iterative tor the .success of tho .ra„, and for its 

potential iapact on thousands o£ adults, for it to I:, continued until the 
unanswered and presently unanserable questions have been explored. If RLS 
„erc to ...evelop texts ol manuals for assessors and . lients on the basis of 
e>:perienee through June im. they would be appropriate for Syracuse, but of 
U=,ited value elsewhere. Therefore. RIS wishes to continue the demonstration 
i„ ,. v,..t,,.ty of settings, with diverse population ,-,ro„p,. to refine the ass>ss- 
„ent cc.=pe„e„ts cf the prosra., .and to ensure its transrer,,hil ity . 
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APPENDIX A 
COMPETENCY nEVELOPMENT TASK FORCE. 



Nina Ahler 

925 Seventh North Street 
Liverpool, New York 13088 
451-4197 

T-House Coordinator 

Syracuse City School District 

William D. Baldwin 

K. D. No. 4, Westmore Lane 

Cortland, New York 13045 

607/756-6824 

Cortland Senior High School 

.Tames J. Bigsby 
236 Coleridge Avenue 
Svracuse, New York 13204 
4,; 6-1575 

Occupational Learning; Center 

Michael L. Borelli 

R. n. No. 2, Saulsbury Road 

Tuliv, New Yok 13159 

696-8412 

Cortland Junior High School 

i.itricia M. Coolicnn 

300 Audubon Parkway, Apartment AlU 

Syracuse, New York 13224 

446-3124 

Futures Institute (Part-Time) 

:)cr>f.-rt U. Daley 

4607 Brookhill Drive South 

Manilas, New York 13104 

682-5616 

.lamesville-Dewitt High School 

He; Ion M. Dierks 

11 : Dorxaar Drive 

North Syracuse, New York 13212 

A 58- 3 109 

i,eague of Women Voters 



Marguerite G. Johnson 
114 Lincoln Avenue 
Syracuse, New York 1320a 
478-7053 

Prescott School (Evenings) 



Peter K. Kavana^h 
147 Hastings Place 
Syracuse, New York 13206 
475-0554 . . u 1 

Roosevelt Junior High School 

Virginia Lewis 

1001 West High Terrace 

Syracuse, New York 13219 

Geo'rgf Sashinftton School Caree; Center 

Patricia Lotito 
120 Scotholm Terrace 
Syracuse, New York 
446-6798 

Salt City Playhouse 

John H. Romano 

5068 Briarl.Ki^e Road 

North Syracuse, New York 

458-8082 

JamesviUe-Dewitt H.;^h School 

Polly Spedding 
R. D. No. 1 

Geneva, New York 1A456 

798-1204 . „ 

Ontario County Cooperative Extension 

Herb T. Williams 

2209 East Fayette Street 

Syracuse, New York 

446-2465 

Folk Art Gallery, Director 
Afro-American Study Center Vacuity 
Syracuse University 
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APPENDIX B 

REGIONAL COMMITTEE FOP 
THE rv^^T PAT. ^JF.W YORK EXTERNAL HIGH SCHOOL DI PLOMA PROGRAM 



CHAIRMAN 



Paul Haley 
District Superintendent 
Cayuga County BOCES 
234 South Street Road 
Auburn, New York 13021 



MEMBERS 

Louis Aiteri 

Manager, Foreman Relations 
General Electric Company 
Building 2, Room 205 
Electronics Parkway 
Syracuse » New York 

Dr. Daniel L. Apling 

Principal on Special Assignment 

to the Director of Occupational 

and Continuing Education 
Syracuse City School District 
A09 West Genesee Street 
Syracuse, New York 13201 

Marian Z. Beauchamp, Principal 
Camillus Junior High School 
Ike Dixon Road 
Camillus, New York 13031 

Peter Blomeriey 
Dean of the College 
Thompkins-Cortland Community College 

170 North Street 
Dryden, New York 13053 

Robert Bond, Teacher 
Hamilton Central High School 
Hamilton, New York 13346 

Ann V. Gillick, Counselor 
Liverpool High School 
901 Walnut Avenue 
Syracuse, New York 



Irvin E. Henry, Executive Oilicer 
Onondaga-Madison Counties BOCES 
6820 Thompson Road 
Syracuse, New York 13211 

Dr. James R. Knox 
Superintendent of Schools 
Education House 
130 South Street 
Auburn, New York 13021 

Judy Nommik, Counselor 
Cortland County BuCES and Schools 
135 Westminster Avenue 
Syracuse, New York 13210 

Arthur Shedd 

Superintendent of Schools 
Cortland City Schools 
Cortland, New York 130A5 

Samuel Villani, Secretary-Treasurer 
Local No. 200 

Service Employees International 
P. 0. Box 1200 
Syracuse, New York 13201 

Barbara Wheler 

Member, Board of Education 

Cazenovia Central School District 

4735 Ricige Krad 

Cazenovia, Mew York 13035 
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EX-OFFICIO MEMBERS 

Francis U. Macy. Director ^ J°7u«rorCo„tlnulng Education 

Regional Learning Service i^ateEducatloa Department 

.of r S-eef" 
Syracuse, New York 13203 
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